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Abstract 

This study used log-linear modeling to examine responses of high school students enrolled in an 
interactive video program during the 1995, 1996, 1997, and 1998 program years to open-ended 
questions concerning strengths and weaknesses of the program. Respondents were categorized by 
home or remote site and by semester. Analyses of the strength responses provided a model with a 
relationship between strength and site and between strength and semester. The weakness 
response required a model including the interaction of weakness, site, and semester. Results are 
discussed. 
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High School Student Perceptions of the Strengths and Weaknesses 
of an Educational Interactive Video Program over a Four-Year Time Span: Log-linear Analysis 

Although interactive video technology has advanced rapidly in recent years, there is 
increasing evidence that no one technology works in every application (Linking, 1989). In some 
areas differences in culture may dictate different learning methods. Differences in grade and/or 
age may require different teaching styles. In addition, the technology utilized by interactive 
video requires a different preparation for teaching than traditional methods (Knapczyk, 1993). 
While the expenditure required for interactive video may be less than providing on site teachers 
(Morgan, 1994; Villarroel, 1988) or may provide access to equal educational opportunities in 
rural areas, there is an increasing need for research into and evaluation of the effectiveness of 
interactive video programs. In 1989 Prideaux noted that effectiveness was not adequately 
evaluated. Madan (1997) added that the credibility of interactive video distance learning is 
questionable unless the quality of the program is assessed. Thus, the need for evaluation of 
distance education programs seems clear. 

Most studies, however, that could be found evaluated a single semester or year and for a 
single class. Thus, while effectiveness may be evaluated for a short period of time and over an 
individual class, results for an extended time with multiple classes are still sparse. 

The primary purpose of this study was to evaluate the strengths and weaknesses of an 
interactive video program in a rural Appalachian area in southwest Virginia as perceived by high 
school student participants. Because attitude is a reflection of the perceived effectiveness of the 
program, prior results have been used to provide improvements to the program. This analysis 
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continued an assessment of change over the 4-year life span of the program. 

Because the method of analysis influences the interpretation of results, two researchers 
may provide different answers to the same question. Thus, a secondary purpose of this research 
was to compare the use of log-linear modeling with categorical data to the use of the chi-square 
test of independence. 

Literature Review 

Distance Education 

The basic criterion for distance education is distance between the teacher and the student. 
Distance education is not a new technique. This technique was begun in the nineteenth century 
with correspondence education (Klesius, Homan, & Thompson, 1997). It has, however, changed 
from the correspondence delivery method, through radio methods, to today’s computer and 
interactive video techniques. 

Increased use of interactive video (ITV) for distance learning students has made classes 
previously inaccessible to rural high school students available locally. Offering college courses 
at remote sites by interactive video permits students to take the class without a lengthy drive and, 
in some localities, homebound students may now partake of classroom interaction by interactive 
video. Thus, we may perceive interactive video as a means of providing equal educational 
opportunities to all students. There are, however, questions concerning this program. 

Prior researchers in distance education have investigated student satisfaction, 
communication techniques, teaching behavior, and change fostered (Moore & Thompson, 1990). 
Because distance education places students in the situation in which there may be no interaction 
or association with other students or the teacher, system requirements must be sound 
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(Gunawardena, 1988; as cited in Dillon, Gunawardena, & Parker, 1992). In addition, the 
importance of the role of the teacher or facilitator has been stressed by several researchers 
(Garrison & Baynton, 1987 as cited in Dillon, Gunawardena, & Parker, 1992) and interaction 
with the instructor has been central to the success of a distance education program. When a 
distance education program has active support, some researchers have found no differences in 
program rating between home and remote sites. Thyer, Polk, and Gaudin (1997), however, 
reported that live instruction was rated significantly higher at a college campus than distance 
learning. They add that distance learning has not yet demonstrated comparable outcomes in terms 
of student learning. 

Although the use of distance education provides the obvious advantage to take otherwise 
unavailable classes, as the role of distance learning expands, it is essential that the problems 
unique to this format be examined (Wilson, Litle, Coleman, & Gallagher, 1997/98). What do 
students perceive as advantages and disadvantages of the distance education program? How do 
programs change over time? There is also the traditional problem of how to analyze the data 
collected. 

Data Analysis 

In analysis of open-ended questions, many times responses are summed to categories and 
the categories contrasted by chi-square (x 2 ) goodness of fit. Categories may then be cross- 
classified by another factor and independence of variables tested by the x 2 test of independence. 
Categories may again be cross-classified by a second factor and the process repeated. This 
procedure leads to multiple significance tests and provides no method to assess interaction of 
more than two variables. 
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Fienberg (1989) says that “the use of multiple two-dimensional tables to analyze data 
fails to distinguish the relationship when other variables are present, does not permit 
simultaneous examination of the relationship, and ignores higher-order interactions” (p. 1). In 
addition, the use of multiple tests of significance inflates the type I error rate. Log-linear 
modeling, on the other hand, permits the user to assess interaction between multiple variables. 
Thus, a log-linear model provides a better measure to assess multidimensional categorical data. 

Bakeman and Robinson (1994) compare the use of a log-linear model to hierarchical 
multiple regression. Regression predicts scores for an individual. Log-linear modeling predicts 
scores for a cell. The degrees of freedom for regression are based on number of subjects. The 
degrees of freedom in log-linear modeling are based on number of cells as is x 2 - Both, however, 
may use multiple predictors to build the model. 

Method 

Subjects 

High school students enrolled in an interactive video class at a distance education facility 
in a rural southwest Virginia area were surveyed during the Spring semesters, 1995, 1996, and 
1997. During the 1997-98 school year, surveys were submitted to students during both the Fall 
and Spring semesters. Surveys were administered during the regularly scheduled class time by 
the class instructor or remote site facilitator. 

Measurement 

The original survey instrument consisted of demographic information (school, gender, 
grade, etc.), some 5-point Likert style questions, and three open-ended questions concerning 
strengths, weaknesses, and suggested improvements for the interactive video program. Two of 
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the open-ended questions (strengths and weaknesses) were used for this analysis. 

Each of the open-ended questions was followed by three blank lines indicating each 
respondent could provide three answers. Survey responses to the strength and weakness 
questions during the 1 995 and 1 996 semesters were coded based on the response. After 
determining similarities of the responses, these were placed in categories. For example, the 
strength’s response “previously unavailable class” was coded as ‘8'. It and the responses “less 
driving” and “classes in my area” were then summed to a major category “access.” This resulted 
in five categories of response. 

In an attempt to improve the response rate, for the 1996-97 and 1997-98 semesters, the 
open-ended questions were changed to provide a selection of five previously determined 
categories. Respondents were requested to rank the most important categories with 1 representing 
the most important. The other category was followed by blank lines to enable the respondent to 
add any needed suggestions. 

Although a respondent could provide three answers for each of the open-ended questions, 
for this analysis, each response was considered to be independent. That is, each individual 
response to the open-ended questions is the unit of analysis - not the individual respondent. 
Analysis 

This data was analyzed using hierarchical log-linear modeling. In this procedure, a fit is 
produced for estimating the number of respondents in each category. For example, if site is a 
good predictor of number of respondents for the categories of strength (access, student 
interaction, learning, student behavior, and other), site is included in the model. Likewise, if 
semester is a good predictor of strength category, it is included in the model. These results could 
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also be obtained using a chi-square test of independence. Site, semester, and strength, however, 
could not be included in the same model. Thus, we could not test interactions between site, 
semester, and strength. The log-linear model tests this interaction. 

Results 

Descriptives 

There were 1024 respondents included in the high school comparison for the four survey 
years in this analysis. Thirty-three schools were included. Six hundred twenty-five respondents 
were from home site locations (95 n=66, 96 n=160, 97 n=86, Fall 97 n=85, 98 n=228 ) and 397 
from remote site locations (95 n=40, 96 n=77, 97 n=61, Fall 97 n=63, 98 n=156) with 2 not 
reporting site (see Figure 1). Respondents were from four grade levels: 8th and 9th grade 
(n=202), 10th grade (n=183), 11th grade (n=246), and 12th grade (n=384) with 9 not reporting 
grade. The prevalent reason for taking interactive video classes has been the respondents wanted 
the class (N=406) or thought it would be interesting (N=246). At the home site, advisors 
frequently suggested the class (N=226). 



Insert Figure 1 About here 



Strengths 

Nine hundred thirty-six of the high school respondents reported one or more strengths of 
the interactive video program for a total of 2327 responses. Of these 1419 were home site 
responses. Perceived strengths of the interactive video program have differed by site during each 
semester. Typically, access to classes has been cited as a strength by remote site respondents 
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more frequently while sociability with other students has been the prevalent strength designated 
by home site respondents (see Figure 2). Over the five semesters, however, the frequency of 
specifying access has decreased proportionally by respondents at the remote site while the 
frequency of citing student behavior and learning have increased. In addition, the proportion of 
responses citing sociability has decreased at the home site while the proportion of responses 
citing student behavior has increased. 



Insert Figure 2 About here 



For the analysis of Strength, the best fitting model (x 2 =25.61, df=16, p=.06) contained 
the univariate terms site, strength, and semester, as well as the interaction terms, site by strength, 
site by semester, and semester by strength. There was no interaction between site, semester, and 
strength. 

Across semesters Student Behavior was cited as a strength with increasing proportional 
frequency. A standardized residual [SR] of ±2 was used to determine statistically significant 
changes. In 1995 (SR= -3) and 1996 (SR= -4.6), student behavior was cited significantly less 
frequently than expected. In the Spring semesters, 1997 (SR=2.1) and 1998 (SR=2.6), student 
behavior was cited more frequently than expected (see Figure 3). In 1995 (SR=2.7) and 1996 
(SR=3.6), sociability with other students was cited significantly more frequently than in other 
years. Learning and access were consistently cited as strengths by 20% to 30% of the responses 
across survey semesters. Other strengths cited were less than 10% for each semester. 
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Insert Figure 3 About here 



Home site responses (SR=-2.2) specified access to classes as a strength less frequently 
than expected while remote site responses (SR=2.7) cited access more frequently them expected. 
On the other hand, home site responses referred to sociability with other students (SR=2.1) with 
greater frequency than remote site responses (SR-2.7). Learning (28%), student behavior (15%), 
and other responses were cited with relatively equal proportional frequency regardless of site (see 
Figure 4). 



Insert Figure 4 About here 



Because there was no interaction between site, semester, and strengths, if strengths had 
been analyzed by two chi-square tests of independence, the results would have been similar. 
Weaknesses 

Eight hundred ninety -two of the high school respondents reported one or more 
weaknesses of the interactive video program for a total of 2044 responses. Of these, 1203 were 
home site responses. Perceived weaknesses of the interactive video program in 1995 were 
dominated by equipment (53%). This weakness has decreased over the five survey semesters. 
The proportional frequency of weaknesses reported for scheduling, on the other hand, increased 
through 1997. Student Behavior and Instruction have increased and decreased depending on 
semester and site (see Figure 5). 
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Insert Figure 5 About here 



When weakness was analyzed by a log-linear model, there was a three-way interaction 
between site, semester, and weakness reported. If forced to a model containing only three 2-way 
interactions, the proportion of responses citing equipment as a weakness was significantly larger 
at the home site in 1995 (SR= 5.16) and 1996 (SR= 4.06) and the remote site in 1995 (SR= 2.25) 
than at other times (see Figures 6 and 7). Although the total proportion of respondents citing 
equipment problems has decreased across survey semesters, there has been a slight increase 
following the Spring 1997 semester at both sites. 



Insert Figures 6 & 7 About here 



Weaknesses concerning scheduling problems were infrequently cited in 1995 (SR home 
=-3.9; SR^,^ = -3.6) and in 1996 (SR home =-3.4; SR remote = -3.2). By 1997, however, scheduling 
was cited with increasing frequency by both home and remote site respondents. More than 30% 
of the weakness responses at the home site reported scheduling problems in the three semesters 
surveyed in 1997 and 1998 while more than 25% of the remote site respondents reported the 
same. 

At the remote site, the proportional frequency of citing instruction as a weakness was 
higher than at the home site in 1995, increased from 1995 to 1996, and then decreased from 
1996 to Spring, 1997 (SR= -2.5). At the home site, the proportional frequency of citing 
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instruction as a weakness exhibited a random fluctuation decreasing slightly from 1995 to 
Spring, 1997, increasing in Fall 1997, and decreasing in 1998. 

Student behavior weaknesses were reported with low proportional frequency in 1995 (SR 
= -2.2) but increased in 1 996 (SR = 2.32) and Spring, 1997 (SR = 2.09) at the home site. At the 
remote site student behavior weaknesses were reported less frequently from 1995 to 1996 (SR = - 
4) and then increased in Spring, 1997. All categories exhibited a relatively constant proportional 
frequency after the Spring, 1 997, semester. 



Conclusion 

As expected, strengths were perceived differently by home and remote site respondents. 
At the home site sociability or meeting new people was important. At the remote site access was 
important. In addition, the proportion of responses citing student behavior as a strength has 
increased across semesters. The learning and access strengths were cited with relatively equal 
frequency across semesters. 

The weakness responses required a three-way interaction between weakness, site, and 
semester. The proportion citing equipment weaknesses has decreased consistently across 
semesters at both home and remote sites. Obviously there has been improvement in 
maintenance and care of equipment in this program. In addition, scheduling problems are cited 
with increasing frequency across semesters at both sites reaching a proportional peak in Spring, 
1997, and remaining there for the following two semesters. There were, however, differences in 
the instruction and student behavior weaknesses based on semester and site. 

Remote site respondents reported more weaknesses in instruction (interaction with the 
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teacher, 1-on-l with the teacher) in 1996 than in 1995 and 1997. Apparently in 1995 and 
increasingly in 1996, interaction of the teacher with remote site respondents was difficult. By 
1997 this problem had been resolved. Home site respondents, on the other hand, reported 
weaknesses in instruction with decreasing frequency after 1995 and have remained relatively 
stable across semesters. 

Remote site respondents also reported student behavior as a weakness less frequently in 
1996 than in 1995 and 1997. Student behavior at the remote site (cheating, lack of discipline, 
absence) appeared to improve in 1996. In addition, home site respondents reported student 
behavior as a weakness with increasing frequency across semesters. This weakness, however 
continues to be a concern at both home and remote sites. 

Student behavior was considered a strength and a weakness in this analysis. It should be 
noted that responses to the strength category in student behavior were citing better study habits 
and better listening skills. Weaknesses in student behavior cited cheating, discipline, and 
absence. While both responses pertain to a student’s behavior, they represent opposing ends of 
the spectrum 

An increasing proportion of the respondents at both home and remote sites are citing 
scheduling weaknesses across semesters. This is the most frequently cited weakness at both 
home and remote sites in 1 997. Respondents suggest those schools in the same network should 
be on the same schedule and have the same snow schedule. 

It was also concluded that the use of log-linear modeling provided a more appropriate 
method of analysis for this data. If weaknesses had been analyzed by two % 2 tests of 
independence — one site by weakness, and one semester by weakness - results would have 
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differed. The only difference between sites would have been in instruction with the home site 
reporting instruction as a weakness less frequently than expected and the remote site reporting 
instruction more frequently than expected. Across semesters, Equipment would have been cited 
more frequently than expected in 1995 and 1996. Student Behavior would have been reported as 
a weakness more frequently than expected in 1997 than less than expected in 1995. The 
discrepancy in Student Behavior between home and remote sites in 1995 and 1996 would not 
have been noticed. In addition, Instruction would have been reported as a weakness less 
frequently in 1997 and more frequently in 1996. Instruction weakness citations have been 
relatively stable proportionately at the home site. Only the remote site has shown significant 
changes. This would not have been detected in % 2 analyses. 

Another problem noted in this analysis was concerned with the "open-ended" questions 
about strengths and weaknesses of the program. These questions were converted to choices on 
the 1997 survey instrument with the respondent rating the top three strengths or weaknesses 
using the categories provided. There was a corresponding increase in number of responses. 
However, it is unknown whether the respondent believed these responses were worthy of notice, 
or if the respondent was simply cooperating by answering the questions. Even though an ‘other’ 
category with blank lines was given, it was seldom used. Future surveys should return to the 
open-ended blank lines. Then a response is written only if the respondent believes it needs to be 
included and, no responses are suggested by the instrument. 
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Strengths (HS Students 1995-98) 

Proportional by Semester 




Figure 4 Strengths (HS Students 1995-98) 

Proportional by Site 
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Figure 5 Weaknesses (HS Students 1995-1998) 

(Proportion by Site/Semester) 
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Figure 7 Weaknesses (Remote Site 1 995-98) 

HS Students - Proportional by Semester 
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